.Dd December 5, 2023
.Dt LLAVA-QUANTIZE 1
.Os Llamafile Manual
.Sh NAME
.Nm llava-quantize
.Nd CLIP model quantizer
.Sh SYNOPSIS
.Nm
.Op options...
.Ar INPUT
.Ar OUTPUT
.Ar FORMAT
.Sh DESCRIPTION
.Nm
makes LLaVA mmproj files smaller.
.Sh ARGUMENTS
The following positional arguments are accepted:
.Bl -tag -width indent
.It Ev Ar INPUT
Is the input file, which should be a CLIP model in the GGUF format using float16 values.
.It Ev Ar OUTPUT
Is the output file, which will be a CLIP model in the GGUF format using the desired number type.
.It Ev Ar FORMAT
Is the desired quantization format, which may be the integer id of a supported quantization type. See the quantization types section below for acceptable formats.
.El
.Sh OPTIONS
The following options are accepted:
.Bl -tag -width indent
.It Fl h , Fl Fl help
Show help message and exit.
.It Fl Fl version
Print llamafile version.
.El
.Sh QUANTIZATION TYPES
The following quantization types are available:
.Pp
.Bl -dash -compact
.It
2 is Q4_0
.It
3 is Q4_1
.It
6 is Q5_0
.It
7 is Q5_1
.It
8 is Q8_0
.El
.Sh SEE ALSO
.Xr llamafile 1
